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ABSTRACT 

Prognostics  is  an  emerging  science  of  predicting  the 
health  condition  of  a  system  (or  its  components)  based 
upon  current  and  previous  system  states.  A  reliable 
predictor  is  very  useful  to  a  wide  array  of  industries  to 
predict  the  future  states  of  the  system  such  that  the 
maintenance  service  could  be  scheduled  in  advance 
when  needed.  In  this  paper,  an  adaptive  recurrent  neural 
network  (ARNN)  is  proposed  for  system  dynamic  state 
forecasting.  The  developed  ARNN  is  constructed  based 
on  the  adaptive/recurrent  neural  network  architecture 
and  the  network  weights  are  adaptively  optimized  using 
the  recursive  Levenberg-Marquardt  (RLM)  method. 
The  effectiveness  of  the  proposed  ARNN  is 
demonstrated  via  an  application  in  remaining  useful  life 
prediction  of  lithium-ion  batteries. 

1.  INTRODUCTION 

Prognostics  and  health  management  (PHM)  is  an 
enabling  discipline  of  technologies  and  methods  with 
the  potential  capability  of  solving  reliability  problems 
that  have  been  manifested  due  to  complexities  in 
design,  manufacturing,  environmental  and  operational 
conditions  (Pecht  and  Jaai,  2010).  Although  a  key 
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component  of  PHM  is  prognostics,  it  is  also  the  least 
mature  element  under  development  (Goebel  et  al., 
2008).  Prognostics  entails  the  use  of  the  current  and 
previous  system  (or  component)  states  to  predict  the 
future  states  of  a  dynamic  system,  and  the  forecast 
information  can  be  used  for:  1)  condition  monitoring  to 
provide  a  reliable  alarm  before  a  fault  reaches  critical 
levels,  so  as  to  prevent  machinery  performance 
degradation,  malfunction,  or  even  catastrophic  failures 
(Liu  et  al.,  2010);  and  2)  schedule  of  repairs  and 
maintenance  in  advance  (Jardine  et  al.,  2006). 

Traditionally,  prognostics  can  be  implemented  using 
either  model-based  approaches  or  data-driven 
approaches  (Schwabacher  and  Goebel,  2007).  Model- 
based  approaches  typically  involve  building 
mathematical  functions  to  describe  the  physics  of  the 
system  and  failure  modes  and  thus  incorporate  physical 
understanding  of  the  system  into  the  estimation  of 
remaining  useful  life  (RUL)  (Saha  and  Goebel,  2009; 
Orchard  et  al.,  2005;  Oppenheimer  and  Loparo,  2002; 
Adams,  2002;  Chelidze  and  Cusumano,  2004). 
However,  an  accurate  analytical  model  is  usually 
difficult  to  derive  for  a  complex  dynamic  system, 
especially  when  the  system  operates  under  noisy  and/or 
uncertain  environments  (e.g.,  most  real-world  industrial 
applications).  Data-driven  approaches  use  pattern 
recognition  and  machine  learning  to  establish  the 
connection  between  operational  data  and  system  health, 
thereby  enabling  changes  in  system  states  to  be 
detected  (Tse  and  Atherton,  1999;  Gupta  and  Ray, 
2007;  Goebel  et  al.,  2008).  Classical  data-driven 
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approaches  for  nonlinear  system  prediction  also  include 
the  use  of  stochastic  models  such  as  the  autoregressive 
(AR)  model  (Groot  and  Wurtz,  1991),  the  threshold  AR 
model  (Tong  and  Lim,  1980),  the  bilinear  model 
(Subba  Rao,  1981),  the  projection  pursuit  (Friedman 
and  Stuetzle,  1981),  the  multivariate  adaptive 
regression  splines  (Friedman,  1991),  and  the  Volterra 
series  expansion  (Brillinger,  1970).  These  approaches 
rely  on  past  patterns  of  the  degradation  of  similar 
systems  to  project  future  system  states  and  are  more 
appropriate  when  the  understanding  of  first  principles 
of  system  operation  is  not  comprehensive  and/or  when 
the  system  is  sufficiently  complex  that  developing  an 
accurate  model  is  prohibitively  expensive.  Since  the 
last  decade,  more  research  interests  in  system  state 
forecasting  have  shifted  to  the  use  of  data-driven 
flexible  models  such  as  neural  networks  (Tse  and 
Atherton,  1999;  Atiya  et  al.,  1999)  and  neural  frizzy 
systems  (Jang,  1993;  Korbicz,  2004). 

Although  data-driven  flexible-model  approaches 
have  demonstrated  some  superior  properties  to  other 
classical  forecasting  tools,  advanced  research  still  needs 
to  be  conducted  before  it  can  be  applied  to  real-world 
industrial  applications  (Wang,  2007):  1)  improving  the 
convergence  properties,  especially  for  multi-step 
predictions;  and  2)  enhancing  the  adaptive  capability  to 
accommodate  time-varying  system  conditions. 
Consequently,  the  aim  of  this  paper  is  to  develop  an 
adaptive  recurrent  neural  network  (ARNN)  for 
prognostic  applications.  The  developed  ARNN 
predictor  is  new  in  the  following  aspects:  1)  a  multi¬ 
layer  state  adaptive  and  recurrent  paradigm  is  proposed 
for  multi-step  forecasting,  such  that  the  information 
from  the  previous  steps  could  be  properly  utilized  to 
improve  the  forecasting  accuracy;  2)  a  recursive 
Levenberg-Marquardt  (RLM)  algorithm  is  adopted  to 
adaptively  optimize  the  network  parameters  to 
accommodate  time-varying  system  conditions;  3)  the 
proposed  ARNN  is  implemented  for  the  RUL 
prediction  of  lithium-ion  batteries.  The  useful  life  of  a 
battery  is  defined  by  the  time  until  which  a  battery  is 
able  to  maintain  a  minimum  charge  capacity  when  frilly 
charged.  In  this  experiment  data  were  collected  at  fixed 
time  intervals  (every  four  weeks),  therefore,  RUL  is 
calculated  as  the  time  remaining  until  the  battery 
charge  retention  capacity  drops  to  70%  of  its  full 
capacity  observed  at  the  beginning. 

The  organization  of  this  paper  is  as  follows.  The 
developed  ARNN  predictor  is  described  in  Section  2. 
The  effectiveness  of  this  ARNN  predictor  is 
demonstrated  in  Section  3  via  an  application  in  battery 
RUL  prediction.  Some  conclusions  and  remarks  are 
included  in  Section  4. 


2.  THE  ADAPTIVE  RECURRENT  NEURAL 
NETWORK 

The  developed  ARNN  predictor  includes  two  main 
components:  the  adaptive  and  recurrent  neural  network 
architecture  and  the  network  parameter  optimization 
using  a  RLM  method,  which  will  be  described  in  the 
following  discussions. 

2.1  The  ARNN  Network  Architecture 

The  network  architecture  of  the  proposed  ARNN 
predictor  is  schematically  shown  in  Figure  1.  It  is  a 
hybrid  model  of  both  direct,  adaptive,  and  recursive 
predictions;  that  is,  the  ARNN  predictor  is  constructed 
based  on  a  feed-forward  multi-layer  neural  network 
with  adaptive  and  recurrent  feedback  links  from  user- 
selected  nodes.  The  connection  topology  may  vary 
from  one  application  to  another,  which  can  be 
conveniently  redefined  by  users.  The  adaptive  feedback 
links  represent  temporal  information  spatially  while  the 
recurrent  feedback  links  deal  with  time  explicitly.  The 
feedback  units  copy  the  activations  of  the  nodes  from 
the  previous  time  step,  and  allow  the  network  to 
memorize  the  clues  from  the  past,  which  forms  a 
reasoning  base  for  current  processing. 


Figure  1:  The  network  architecture  of  the  developed 
ARNN  predictor. 

In  the  developed  ARNN  predictor,  the  sigmoid 
activation  functions  (AF)  are  used  for  the  nodes  in  the 
hidden  layers  and  the  linear  AF  is  chosen  for  the  nodes 
in  the  output  layer;  correspondingly  the  inputs  x;(n), 
the  previous  states  from  the  hidden  layers  yh  (ii  - 1)  , 
and  the  previous  outputs  ya(n-l)  should  be  properly 
normalized  to  [0  1]  or  [-1  1]  based  on  the  type  of  the 
sigmoid  AFs  employed.  For  ra-th  sample,  the  output  of 
the  j-th  node  in  the  first  hidden  layer  is  given  by 

y}(n)  =  f(netj),  (1) 

netj  ( n )  =  £  ujhyh  (n  - 1)  +  £  (n) 

h  i  (2) 

+  Lwjiyo(n-l)+8j . 

/ 
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where  6j  is  the  bias;  U,  V  and  W  are  the  network 
weight  matrices  that  correspond  to  the  previous  states, 
the  inputs  and  the  previous  outputs,  respectively;  1=1, 
2,  L,  where  L  denotes  the  maximum  output 
feedback  depth;  /(•)  is  the  chosen  AF.  The 
connections  between  the  first  hidden  layer  and  the 
network  output  can  take  any  form  of  multi-layer 
perceptron  (MLP),  denoted  by 

yQ(n) = s(y(n)) ,  (3) 

where  g(')  represents  the  nonlinear  mapping  of  this 
MLP.  The  proposed  state  adaptive  and  recurrent 
feedbacks  include  temporal  aspects  into  network  such 
that  the  useful  information  from  the  previous  steps 
could  be  properly  utilized  for  a  more  accurate 
estimation  on  the  future  system  states.  The  network 
weights  are  optimized  by  the  developed  RLM  method 
(Liu  et  al.,  2009)  as  discussed  in  the  following  section. 

2.2  The  ARNN  Network  Training 

The  ARNN  predictor  as  developed  in  Section  2.1 
should  be  properly  trained  to  generate  an  optimal 
input/output  mapping.  For  offline  training, 
representative  training  data  should  cover  all  the 
possible  application  conditions  (Liu  et  al.,  2009).  Such 
requirement  is  difficult  to  achieve  in  real-world 
applications  because  most  machines/systems  operate  in 
noisy  and/or  uncertain  environments.  Usually,  the 
classical  forecasting  schemes  are  employed  for  time- 
invariant  systems  or  systems  with  slowly  varying 
model  parameters.  However,  system’s  dynamic 
characteristics  may  change  suddenly,  for  example,  just 
after  repairs  or  regular  maintenance.  In  this  section,  a 
RLM  method  is  adopted  to  adaptively  optimize  the 
ARNN  weights  to  accommodate  time-varying  system 
conditions. 

For  a  training  data  pair,  jx^  ]  ,  x ^  is  a 
vector  that  contains  the  network  inputs  as  well  as  the 
previous  states  and  outputs;  n  =  1,  2,  N;  N  is  the 
total  number  of  training  data  sets;  is  the  desired 

output.  The  objective  function  for  all  the  N  training 
data  sets  is  defined  as 

(4) 

Z  n= 1  Z  n=1 

where  2e(0  1]  is  the  forgetting  factor, 
and  A  e  (0.9  1]  is  usually  used  to  avoid  possible 
convergence  instability;  ©  denotes  all  the  network 
weights  and  0  e  Rqxl  ;  s  is  the  point  forecasting 
error. 


The  computation  of  the  inverse  Hessian  matrix  in 
the  classical  LM  is  time-consuming  and  impractical  for 
real-time  applications  (Jang  et  al.,  1997).  In  this  case,  a 
remedy  is  to  apply  the  matrix  inversion  lemma  to  avoid 
the  direct  inversion  of  Hessian  matrix  (Ljung  and 
Soderstrom,  1983).  Instead  of  adding  the  qxq  matrix 
/j(n)I  at  each  step,  only  one  diagonal  element  is 
added  at  a  time.  As  a  result,  the  weights  of  the  ARNN 
are  recursively  updated  by 

0(72)  =  0(72-1) 

-  R(n  -l)cp(n, 0(72-1)) T  e(n, ®(n  -1)) 

R(n)  =  -{R(n-l) 

A 

-  R(n-l)(p* (n,®(n-l))T[AA  (n) 

+  q>* (n,®(n  -l))R(n  -l)<p* (n,®(n  -1))T  y1 
(p*  (72,0(72  -  l))R{n  - 1)} 

where  0( n)  is  the  estimate  of  &(n)  at  sample  instant 
rr,(p  =  de/d®  is  the  Jacobian  matrix;  R  is  the  inverse 
of  an  approximated  Hessian  matrix,  and  /?(())  is  chosen 
as  an  identity  matrix  aNI  with  a  constant 

%  e[10”  105  ]  ;  (p  and  A  are,  respectively, 
given  as 


(p  ( 72 ,  0(ti  - 1))  = 

0 

0 

<p(n,®(n  -1))T  1 

0 

0 


position  n  (mod  q)  +1,  (7) 


A  (72)  = 


1  0 

0  qju(n) 


(8) 


The  RLM  parameter  /u(ri)  conversely  affects  the  trust 
region  radius  (Jang  et  al.,  1997),  which  should  be 
adaptively  modified  based  on  the  performance  of  the 
updated  weights.  The  following  strategy  is  applied  in 
this  work:  the  initial  value  ju(l)  is  set  to  0.1;  if  the 
objective  function  in  Eq.  (4)  decreases  as  the  weights 
are  updated,  reduce  ju(n )  by  ju(n)/k',  otherwise,  if  the 
objective  function  increases  as  the  weights  are  updated, 
enlarge  the  ju(n)  by  k/i( n)  ,  where  k  is  a  design 
parameter.  In  determining  these  LM  parameters,  a 
series  of  simulation  tests  have  been  conducted  in  terms 


3 


Annual  Conference  of  the  Prognostics  and  Health  Management  Society,  2010 


of  training  accuracy  and  convergence  speed.  //( 1)  e 
[0.01,  1]  have  been  tested  and  //( 1)  =  0.1  is  used  as  an 
initial  value  which  will  be  updated  in  training.  On  the 
other  hand,  ke  [1.01,  2.00]  was  explored  in  this  work; 
a  larger  value  of  the  updating  parameter  k  may  lead  to 
training  instability,  whereas  a  smaller  k  slows  down  the 
training  process,  k  —  1.15  is  used  in  this  work. 

3.  PERFORMANCE  EVALUATION 

The  effectiveness  of  the  proposed  ARNN  predictor  is 
shown  for  a  battery  health  management  application  for 
RUL  prediction.  We  compare  its  performance  with  the 
classical  recurrent  neural  network  (RNN)  and  the 
recurrent  neural  fuzzy  system  (RNF)  (Liu  et  al,  2009) 
in  more  details  in  addition  to  making  comparisons  with 
other  data  driven  algorithms  that  have  been  applied  to 
the  battery  health  dataset  in  earlier  works. 

3.1  Application:  Battery  RUL  Prediction 

Batteries  are  widely  used  in  various  engineering  and 
household  systems.  An  effective  prognostic  tool  is 
critically  needed  to  predict  the  future  capacity  of  a 
battery  and  reliably  estimate  its  RUL.  The  forecast 
RUL  information  can  be  used  not  only  for  battery  fault 
detection  to  prevent  performance  degradation  of  the 
related  equipment,  but  also  for  scheduling  battery 
recharge  which  is  critical  in  many  applications  such  as 
the  emerging  electric  vehicles  and  aerospace  industries. 
Both  applications  differ  in  the  way  they  define  RUL. 
For  one,  RUL  is  computed  by  tracking  its  charge 
retention  capacity  that  tends  to  diminish  with  every 
recharge  cycle,  and  for  the  other  RUL  is  defined  by  the 
expected  discharge  of  a  battery  within  a  charge  cycle. 
While,  both  are  critical  applications,  the  experimental 
data  used  in  this  paper  is  suitable  for  prediction  battery 
capacity  only. 

Dynamic  models  have  been  built  for  PHM  of 
lithium  ion  batteries  (Gao  et  al,  2002).  These  models 
take  into  consideration  nonlinear  equilibrium 
potentials,  rate  and  temperature  dependencies,  thermal 
effects,  and  transient  power  response.  However,  it  still 
remains  difficult  to  accurately  predict  the  RUL  of  a 
battery  using  a  model-based  approach  when 
environmental  and  load  conditions  changes; 
furthermore,  modeling  a  lithium-ion  battery  from  the 
first  principles  of  the  internal  electrochemical  reactions 
can  be  very  tedious  and  computationally  intractable. 
Developing  data-driven  methods  that  use  sensor 
measurements  to  make  these  predictions  provides 
alternative  ways  to  make  predictions  in  absence  of 
good  system  model.  In  this  work,  we  have  implemented 
the  developed  data-driven  ARNN  predictor  for  the 
RUL  prediction  of  lithium-ion  batteries.  The  battery 
RUL  is  estimated  in  terms  of  capacity  degradation 


(Goebel  et  al,  2008;  Saxena  et  al.,  2009).  The  failure 
threshold  is  generally  defined  by  the  manufacturer  for  a 
specific  application.  In  this  case,  a  lithium-ion  battery 
is  deemed  to  fail  when  its  capacity  C/1  fades  by  30% 
of  the  rated  value.  The  batteries’  capacity  is  usually 
inaccessible  to  be  gathered  though  sensor 
measurements;  therefore,  indirect  measurements  are 
employed  to  assess  battery  health.  A  lumped  parameter 
model  can  be  used  that  models  the  inside  chemistry  of 
the  batteries  though  a  simple  electrical  circuit  and  the 
change  in  resistive  components  of  this  circuit  explains 
the  reduction  in  battery  capacity.  Features  extracted 
from  sensor  data  of  voltage,  current,  power,  impedance, 
frequency,  and  temperature  readings  are  used  to 
estimate  the  internal  parameters  in  the  lumped- 
parameters  battery  model.  The  values  of  these  internal 
parameters  change  with  various  aging  and  fault 
processes  like  plate  sulfation,  passivation,  and 
corrosion  (Goebel  et  al,  2008).  Specifically, 
parameters  RE  and  RqE  are  tracked  and  employed  for 
battery  RUL  prediction,  where  RE  denotes  the 
electrolyte  resistance  and  RCT  denotes  the  charge 
transfer  resistance  in  the  lumped-parameter  model 
(Saha  et  al.,  2009).  ( RE  +  RCT)  is  typically  inversely 
proportional  to  the  capacity  C/1  and  can  be  estimated 
through  the  electrochemical  impedance  spectroscopy 
test.  The  data  used  in  this  study  were  from  the  second 
generation,  Gen  2,  18650-size  lithium-ion  cells  that 
were  cycle-life  tested  at  60%  state-of-charge  (SOC) 
and  temperature  (25°C  and  45°C)  (Christophersen  et 
al,  2006). 

3.2  Results  and  Discussion 

The  prediction  algorithms  developed  in  this  study 
namely;  the  ARNN  predictor,  the  classical  RNN 
predictor  and  the  RNF  predictor  are  trained  using  the 
history  ( RE  +  Rcr)  data  collected  from  multiple 
batteries  of  the  same  type  at  25°C  and  45°C,  and  are 
then  employed  to  predict  an  unknown  ( RE  +  RCT) 
trajectory  of  the  test  battery  collected  at  45°C.  Figure  2 
shows  the  (Re  +  Rct)  trajectories  that  were  used  for 
network  training  and  testing.  As  can  be  seen  from  the 
figure  below,  these  batteries  take  more  than  a  year  to 
age.  Running  such  systems  to  failure  in  a  lab 
environment  could  be  a  lengthy  and  rather  costly 
process.  Therefore,  data  from  multiple  batteries  being 
aged  in  parallel  was  collected  on  a  monthly  basis. 
However,  the  ( RE  +  RCT  )  data  available  in  this  work 
for  network  training  are  quite  limited  in  the  sense  it  has 
very  few  points  (one  per  month  or  less  than  twenty  per 
aging  battery)  to  let  the  network  train  properly  and 
hence  could  not  be  directly  used  for  early  RUL 
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predictions;  thus  data  interpolation  was  applied  to 
generate  more  data  points  for  a  proper  network  weights 
optimization. 


Figure  2:  The  battery  parameter  data  used  for  network 
training  and  testing. 

The  learning  process  on  the  degradation  patterns 
carried  out  from  the  history  (Re  +  RCT )  data  is 
somewhat  complicated.  As  seen  in  Figure  2,  various 
trajectories  obtained  from  experimental  data  are  shifted 
that  results  in  different  target  values  for  a  given  time 
input  during  learning  phase.  Therefore,  the  same  input 
data  (i.e.,  the  previous  degradation  states),  the  training 
targets  (i.e.,  the  one-step-ahead  state)  may  be 
significantly  different,  which  confuses  the  learning  of 
the  predictors  and  thus  necessitate  more  inputs  and 
finer  training  scenarios.  The  initialization  settings  for 
the  parameters  associated  with  the  proposed  RLM 
learning  algorithm  are  based  on  the  literature,  which  is 
a  common  practice  for  most  of  learning  algorithms 
(e.g.,  the  initial  settings  on  weights  and  learning  rate  in 
steepest  gradient).  During  our  testing  phase,  we  found 
that  the  RLM  algorithm  is  very  stable  in  terms  of 
convergence  and  repeatability  when  the  proper  ranges 
of  learning  parameters  are  specified  and  therefore 
impart  higher  degree  of  confidence  in  the  prediction 
results. 

In  our  implementation,  the  temperature  was  used  as 
an  additional  network  input.  As  indicated  earlier  that 
the  experimental  data  was  available  from  tests 
conducted  at  different  temperatures  and  that  the 
operational  temperature  significantly  affects  the 
capacity  life  of  a  battery,  a  prediction  should  take  into 
account  such  operational  environmental  conditions. 
This  facilitates  prediction  of  battery  capacity  at 
different  temperatures.  Several  initial  runs  are  taken  to 
examine  the  complexity  (or  nonlinearity)  of  the  data  so 
as  to  help  define  the  structure  of  the  related  NN -based 
predictors.  It  must  be  noticed  that  designing  a  suitable 
network  is  usually  considered  an  art  with  few 
guidelines  available  from  the  literature.  For  a  fair 


comparison,  the  ARNN  and  RNN  predictors  were  built 
with  the  same  number  of  layers  and  nodes,  that  is,  six 
nodes  for  the  first  hidden  layer,  eight  nodes  for  the 
second  hidden  layer,  and  four  nodes  for  the  third 
hidden  layer.  The  maximum  adaptive  feedback  depth 
was  set  to  four  for  the  ARNN  and  RNN  predictors  and 
was  limited  to  three  for  the  RNF  predictor  (constrained 
by  the  structure,  Liu  et  al.,  2009).  The  predictors  are 
trained  using  the  history  (Re  +  Rct)  data  of  the 
battery  of  the  same  type  so  as  to  capture  the  patterns  on 
how  the  temperature  x1(n)  ,  the  current  battery 
state  y0  (n)  ,  and  the  previous  battery  states  y{)  (n  -  4) , 
y0(n- 3)  ,  y0(n- 2)  ,  y0(n- 1)  affect  the  one-step- 
ahead  battery  state  yQ  (n  + 1)  .  After  the  learning  step 
the  trained  network  is  used  to  predict  the 
(Re  +  R(t  )  degradation  trend  on  an  unknown  test 
trajectory.  Multiple  predictions  were  generated  starting 
from  week  20  to  week  64  at  an  every  four  weeks 
interval.  The  predictor  was  run  fifty  times  to  obtain  50 
{Re  +  R^t)  trajectories  to  gather  statistical 
significance  that  were  then  averaged  to  compute  the 
final  predicted  trajectory.  Using  a  linear  mapping  from 
(Re  +  Rqt )  t0  battery  capacity  (Saha  and  Goebel, 
2008),  the  battery  capacity  from  the  predicted 
trajectories  was  derived.  As  mentioned  earlier,  the 
threshold  for  determining  the  RUL  was  chosen  at  70% 
of  the  rated  capacity. 

To  quantify  the  prognostic  performance,  the  a-X 
prognostic  metric,  as  defined  in  (Saxena  et  al.,  2010),  is 
employed.  We  first  compare  the  predictions  from  these 
three  predictors  with  the  four  algorithms  that  were 
previously  used  for  the  same  dataset  (Goebel  et  al., 
2008;  Saha  et  al.,  2009;  Saxena  et  al.,  2009).  Figure  3 
shows  the  predicted  means  from  all  seven  different 
approaches  that  include  the  ARNN  predictor,  the 
classical  RNN  predictor,  the  RNF  predictor,  a  Particle 
.Filter  based  predictor,  Gaussian  process  regression 
(GPR),  relevance  vector  machine  (RVM),  and  a  simple 
polynomial  regression  approach.  Since  the  previous 
works  compared  only  the  point  predictions,  a 
comparison  using  the  point  predictions  is  made  here. 
Comparing  these  approaches  in  Figure  3,  following 
observations  can  be  made:  1)  the  proposed  ARNN 
predictor,  the  classical  RNN  predictor  and  the  particle 
filtering  (PF)  based  predictor  provide  a  better 
prognostic  performance  than  the  other  four  predictors 
in  terms  of  prediction  accuracy;  2)  the  performance  of 
the  RNF  predictor  is  not  as  good  as  the  ARNN  and 
RNN  predictor;  this  may  be  attributed  to  the  fact  that 
the  RNF  predictor  is  constrained  by  the  number  of 
network  inputs  (usually  less  than  five),  and  the 
complexity  of  the  RNF  structure  (e.g.,  prognostic  rules 
and  consequent  parameters)  increases  exponentially  as 
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the  number  of  inputs  rises;  based  on  a  limited  number 
of  training  data,  the  neural  network  based  techniques 
could  be  more  suitable  in  this  prognostic  application 
than  the  RNF  to  capture  the  common/hidden 
characteristics  between  the  training  data  and  testing 
data;  3)  the  proposed  ARNN  predictor  outperforms  all 
the  other  predictors  in  this  test  and  performs  within 
90%  accuracy  of  the  true  RUL  since  early  predictions; 
4)  the  test  results  also  indicate  that  the  prognostic 
performance  improves  when  adaptive/recurrent 
feedbacks  from  the  previous  system  states  are  properly 
utilized,  5)  the  implemented  RLM  approach  shows  a 
better  learning  capability  than  the  classical  training 
algorithms. 
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Figure  3:  The  battery  RUL  prediction  comparison 
between  seven  different  prediction  methods. 

To  better  understand  the  prediction  variability 
(uncertainty),  distributions  of  the  RUL  predictions  over 
50  program  runs  were  analyzed  using  the  enhanced 
prognostic  metrics  that  were  developed  in  (Saxena  et 
al.,  2010).  These  results  are  plotted  in  Figures  4-6. 


ARNN  (a=0.1 ,  p=0.5) 


Figure  4:  The  distribution  of  the  RUL  predictions  from 
the  proposed  ARNN  predictor. 

Also  indicated  on  the  plots  are  the  percentages  of  RUL 
distribution  overlap  with  the  10%  accuracy  cone  and 
whether  it  satisfies  the  a-X  metric  at  all  times  according 
to  the  /J-criterion  with  /?  =.05. It  can  be  seen  that  the 
confidence  interval  of  the  RUL  prediction  from  the 
proposed  ARNN  predictor  is  markedly  smaller  than  the 
classical  RNN  predictor  and  the  RNF  predictor,  which 
indicates  that  the  ARNN  predictor  can  provide  a  more 
reliable  RUL  prediction  on  this  dataset. 
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Figure  5:  The  distribution  of  the  RUL  predictions  from 
the  classical  RNN  predictor. 


RNF(a=0.1 ,  p=0.5) 


Figure  6:  The  distribution  of  the  RUL  predictions  from 
the  RNF  predictor. 


4.  CONCLUSION 

In  this  paper,  an  adaptive  recurrent  neural  network 
(ARNN)  has  been  developed  for  remaining  useful  life 
(RUL)  prediction  of  lithium-ion  batteries.  This  ARNN 
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predictor  can  enhance  the  prediction  accuracy  by 
properly  utilizing  the  previous  system  states  through 
adaptive/recurrent  feedbacks.  A  recursive  Levenberg- 
Marquardt  (RLM)  technique  is  implemented  to 
adaptively  optimize  the  ARNN  weights  so  as  to 
accommodate  time-varying  system  conditions.  The 
developed  ARNN  predictor  was  applied  to  predict  the 
remaining  useful  life  (RUL)  of  lithium-ion  batteries. 
The  results  of  this  investigation  have  shown  that  the 
ARNN  technique  can  effectively  learn  system  states 
from  a  limited  number  of  measurements  to  update  the 
data-driven  nonlinear  prediction  model.  It  outperforms 
the  classical  RNN  and  the  RNF  in  battery  RUL 
prediction. 
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NOMENCLATURE 

n  sample  index  (represents  time  for  time  series) 

i  input  index 

j  node  index  in  a  network  layer 

h  layer  index  in  a  network 

l  index  to  denote  output  feedback  depth  (look 
back  distance  in  time) 

6j  bias  for  the  jth  node 
U  weight  matrix  (ujh)  for  previous  states 
V  weight  matrix  (Vji)  for  network  inputs 
W  weight  matrix  ( Wji )  for  previous  outputs 
/( •)  activation  function  (AF) 

g(-)  a  nonlinear  mapping  between  a  hidden  layer 
and  the  network  output  of  a  MLP 
Xi(n)  network  inputs 

yh(n- 1)  previous  states  from  hidden  layer  h 
yo(n-l)  previous  output  looking  back  1  time  instants 
xin)  vector  of  network  inputs  and  previous  states 
and  outputs 

y^l)  desired  output  (target)  for  the  network 

A  forgetting  factor  to  weigh  previous  inputs 
differently  A  e  (0  1] 

©  all  network  weights,  0  e  R‘,xl 
£  point  forecasting  error 
0( n)  is  the  estimate  of  0(ra)  at  sample  instant  n 
(p  is  the  Jacobian  given  by  ds/dQ 

R  is  the  inverse  of  an  approximated  Hessian 
matrix 

oat  is  a  constant  to  initialize  i?;  i?(0)  =  aw/ 
pin)  RLM  parameter  inversely  proportional  to  trust 
region  radius 


k  a  design  parameter  to  scale  p, 

Re  denotes  the  electrolyte  resistance  of  the  battery 
Rct  denotes  the  charge  transfer  resistance  of  the 
battery 

C/1  battery  capacity 

a-A  accuracy  bounds  to  satisfy  while  evaluating 
prediction  performance  at  specific  time 
instant,  A(EoL-tP),  relative  to  ailing  life  of  the 
components 

ft  required  probability  specified  for  RULs  to  fall 
within  a  bounds 
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